Path Planning Problems with Side Observations-When Colonels Play Hide-and-Seek

Resource allocation games such as the famous Colonel Blotto (CB) and Hide-and-Seek (HS) games are often used to model a large variety of practical problems, but only in their one-shot versions. Indeed, due to their extremely large strategy space, it remains an open question how one can efficiently learn in these games. In this work, we show that the online CB and HS games can be cast as path planning problems with side-observations (SOPPP): at each stage, a learner chooses a path on a directed acyclic graph and suffers the sum of losses that are adversarially assigned to the corresponding edges; and she then receives semi-bandit feedback with side-observations (i.e., she observes the losses on the chosen edges plus some others). We propose a novel algorithm, EXP3-OE, the first-of-its-kind with guaranteed efficient running time for SOPPP without requiring any auxiliary oracle. We provide an expected-regret bound of EXP3-OE in SOPPP matching the order of the best benchmark in the literature. Moreover, we introduce additional assumptions on the observability model under which we can further improve the regret bounds of EXP3-OE. We illustrate the benefit of using EXP3-OE in SOPPP by applying it to the online CB and HS games.Comment: Previously, this work appeared as arXiv:1911.09023 which was mistakenly submitted as a new article (has been submitted to be withdrawn). This is a preprint of the work published in Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI

A simple and efficient algorithm to compute epsilon-equilibria of discrete Colonel Blotto games: Extended Abstract

International audienceThe Colonel Blotto game is a famous game commonly used to model resource allocation problems in domains ranging from security to advertising. Two players distribute a fixed budget of resources on multiple battlefields to maximize the aggregate value of battlefields they win, each battlefield being won by the player who allocates more resources to it. Recently, the discrete version of the game-where allocations can only be integers-started to gain traction and algorithms were proposed to compute the equilibrium in polynomial time; but these remain computationally impractical for large (or even moderate) numbers of battlefields. In this paper, we propose an algorithm to compute very efficiently an approximate equilibrium for the discrete Colonel Blotto game with many battlefields. We provide a theoretical bound on the approximation error as a function of the game's parameters. Through numerical experiments, we show that the proposed strategy provides a fast and good approximation even for moderate numbers of battlefields

Combinatorial Bandits for Sequential Learning in Colonel Blotto Games

International audienc

Efficient computation of approximate equilibria in discrete Colonel Blotto games

The Colonel Blotto game is a famous game commonly used to model resource allocation problems in many domains ranging from security to advertising. Two players distribute a fixed budget of resources on multiple battlefields to maximize the aggregate value of battlefields they win, each battlefield being won by the player who allocates more resources to it. The continuous version of the game—where players can choose any fractional allocation—has been extensively studied, albeit only with partial results to date. Recently, the discrete version—where allocations can only be integers—started to gain traction and algorithms were proposed to compute the equilibrium in polynomial time; but these remain computationally impractical for large (or even moderate) numbers of battlefields. In this paper, we propose an algorithm to compute very efficiently an approximate equilibrium for the discrete Colonel Blotto game with many battlefields. We provide a theoretical bound on the approximation error as a function of the game's parameters. We also propose an efficient dynamic programming algorithm in order to compute for each game instance the actual value of the error. We perform numerical experiments that show that the proposed strategy provides a fast and good approximation to the equilibrium even for moderate numbers of battlefields

Efficient computation of approximate equilibria in discrete Colonel Blotto games

The Colonel Blotto game is a famous game commonly used to model resource allocation problems in many domains ranging from security to advertising. Two players distribute a fixed budget of resources on multiple battlefields to maximize the aggregate value of battlefields they win, each battlefield being won by the player who allocates more resources to it. The continuous version of the game—where players can choose any fractional allocation—has been extensively studied, albeit only with partial results to date. Recently, the discrete version—where allocations can only be integers—started to gain traction and algorithms were proposed to compute the equilibrium in polynomial time; but these remain computationally impractical for large (or even moderate) numbers of battlefields. In this paper, we propose an algorithm to compute very efficiently an approximate equilibrium for the discrete Colonel Blotto game with many battlefields. We provide a theoretical bound on the approximation error as a function of the game's parameters. We also propose an efficient dynamic programming algorithm in order to compute for each game instance the actual value of the error. We perform numerical experiments that show that the proposed strategy provides a fast and good approximation to the equilibrium even for moderate numbers of battlefields

Efficient Computation of Approximate Equilibria in Discrete Colonel Blotto Games

International audienceThe Colonel Blotto game is a famous game commonly used to model resource allocation problems in many domains ranging from security to advertising. Two players distribute a fixed budget of resources on multiple battlefields to maximize the aggregate value of battlefields they win, each battlefield being won by the player who allocates more resources to it. The continuous version of the game-where players can choose any fractional allocation-has been extensively studied , albeit only with partial results to date. Recently , the discrete version-where allocations can only be integers-started to gain traction and algorithms were proposed to compute the equilibrium in polynomial time; but these remain computationally impractical for large (or even moderate) numbers of battlefields. In this paper, we propose an algorithm to compute very efficiently an approximate equilibrium for the discrete Colonel Blotto game with many battlefields. We provide a theoretical bound on the approximation error as a function of the game's parameters, in particular number of battlefields and resource budgets. We also propose an efficient dynamic programming algorithm to compute the best-response to any strategy that allows computing for each game instance the actual value of the error. We perform numerical experiments that show that the proposed strategy provides a fast and good approximation to the equilibrium even for moderate numbers of battlefields

Combinatorial Bandits for Sequential Learning in Colonel Blotto Games

International audienc

Approximate Equilibria in Non-constant-sum Colonel Blotto and Lottery Blotto Games with Large Numbers of Battlefields

In the Colonel Blotto game, two players with a fixed budget simultaneously allocate their resources across n battlefields to maximize the aggregate value gained from the battlefields where they have the higher allocation. Despite its long-standing history and important applicability, the Colonel Blotto game still lacks a complete Nash equilibrium characterization in its most general form-the non-constant-sum version with asymmetric players and heterogeneous battlefields. In this work, we propose a simply-constructed class of strategies-the independently uniform strategies-and we prove them to be approximate equilibria of the non-constant-sum Colonel Blotto game; moreover, we also characterize the approximation error according to the game's parameters. We also introduce an extension called the Lottery Blotto game, with stochastic winner-determination rules allowing more flexibility in modeling practical contexts. We prove that the proposed strategies are also approximate equilibria of the Lottery Blotto game